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ABSTRACT 

An  experiment  is  presented  in  which  the  effects  of  roll  motions 
on  human  operator  performance  were  investigated.  The  motion  cues 
considered  were  the  result  of  commanded  vehicle  motion  and  vehicle  disturbances. 
An  optimal  control  pilot-vehicle  model  was  used  in  the  design  of  the  experiment 
and  to  predict  system  performance  prior  to  executing  the  experiment.  The 
model  predictions  and  experimental  results  are  compared.  Seventy-eight  per 
cent  of  the  model  predictions  are  within  one  standard  deviation  of  the  means 
of  the  experimental  results.  The  high  correlation  between  model  predictions 
and  system  performance  indicate  the  usefulness  of  the  predictive  modei  for 
experimental  design  and  for  prediction  of  pilot  performance  influenced 
by  motion  cues. 

INTRODUCTION 

A requirement  exists  in  the  Air  Force  for  a predictive  human 
operator  pilot  model  which  is  sensitive  to  complex  motion  environments. 

Such  a model  would  have  a number  of  important  applications.  For  example,  one 
might  use  the  model  to  (1)  determine  whether  or  not  motion  cues  are  used  by 
the  pilot  in  a particular  control  situation;  (2)  extrapolate  the  results  of 
fixed-base  simulation  to  a motion  environment;  (3)  facilitate  the  design 
of  ground-based  simulators;  (4)  identify  situations  where  misinterpretation  of 
motion  cues  is  likely  to  cause  a pilot  response  that  seriously  degrades  system 
performance.  At  the  Environmental  Medicine  Division  of  the  Aerospace  Medical 
Research  Laboratory  (AMRI.)  , a research  program  is  being  pursued  to  satisfy 
this  requirement.  It  is  directed  towards  developing  a generalized  description 
of  the  manner  in  which  a pilot  uses  motion  cues,  with  the  ultimate  goal  of 
providing  a model  that  can  predict  the  effects  of  motion  cues  on  system 
performance  in  a variety  of  control  situations. 


*Thc  research  reported  in  this  article  was  conducted  by  personnel  of  the 
Aerospace  Medical  Research  Laboratory,  Aero.*; pace  Medical  Division,  Air  Force 
Systems  Command,  Wr lght-Patterson  Air  Force  Base,  Ohio.  The  voluntary 
informed  consent  of  the  subjects  used  in  this  research  was  obtained  as  required 
by  Air  Force  Regulation  80-33.  Further  reproduction  is  authorized  to  satisfy 
the  needs  of  the  U.S.  Government.  Reprints  of  this  article  are  identified  by 
the  Aerospace  Medical  Research  Laboratory  as  AMRL-TR- 7 7- (a 7 • 
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Although  a number  of  experimental  studies  have  been  conducted  to  determine 
the  effect  of  motion  cues  on  pilot  response  behavior  (1-5),  a generalized 
model  has  not  been  developed  and  tested.  Rather,  the  conclusions  reached 
in  these  studies  have  been  restricted  to  the  context  of  the  experiments 
yielding  the  data.  In  addition,  other  than  the  pitch  axis  motion  experiment 
performed  by  vanGool  and  Mooij  (5)  , the  work  done  in  this  area  has 
principally  been  for  compensatory  systems  with  the  motion  cues  resulting 
from  vehicle  disturbance  inputs.  At  AMRL,  we  are  also  interested  in  the 
effects  of  motion  cues  on  performance  resulting  from  commanded  inputs  as 
encountered  in  air-to-air  combat  situations.  Therefore,  a scries  of 
experiments  were  performed  at  AMRL  to  investigate  the  effects  of  motion 
cues  resulting  from  commanded  inputs  on  tracking  performance  (6,7,8). 

One  of  the  products  from  this  research  effort  was  a modification  of 
the  Bolt  Berane.k  and  Newman  optimal  control  pilot-vehicle  model  to  account 
for  changes  in  performance  caused  by  the.  presence  of  motion  cues  due  to 
commanded  inputs.  Since  the  pilot  model,  has  predictive  capabilities,  the  next 
step  was  to  ascertain  how  well  it  could  predict  pilot  performance  under 
different  experimental,  conditions.  The  ability  of  the  model  to  predict  human 
performance  and  aid  in  experimental  design  is  the  topic  of  this  paper. 

A moving  base  simulator,  different  from  the  one  used  in  the  earlier 
series  of  experiments,  was  employed.  Different  vehicle  dynamics  were 
used  and  motion  cues  resulting  from  both  commanded  vehicle  motion  and 
vehicle  disturbances  were  explored.  In  addition,  the  pilot  model  was  used 
to  aid  in  preliminary  experimental  design  to  insure  that  vehicle  motions 
resulting  from  pilot  inputs  would  remain  within  the  linear  operating  range 
of  the  simulator.  After  experimental  conditions  had  been  selected,  the  optimal 
control  pilot-vehicle  model  was  exercised  for  the  various  conditions  and 
performance  scores  were  computed.  Human  tracking  was  then  performed  and 
performance  data  was  collected.  Tire  control  vehicle,  experimental  design  and 
results  from  model  prediction  and  human  tracking  are  presented  and  discussed 
in  this  paper. 

EQUIPMENT 

A Multi-Axis  Tracking  Simulator  (MATS)  was  used  as  the  controlled  vehicle 
for  this  experiment.  Only  the  roll  axis  motion  capabilities  of  the  MATS 
were  used.  The  simulator  consisted  of  a single  seat  cockpit  with  a television 
monitor  display  and  side  mounted  force  stick  for  vehicle  control.  The 
cockpit  was  configured  such  that  the  pilot  sat  one  inch  above  the  rol.1  axis  of 
the  simulator.  The  vehicle  cockpit  was  light-tight  to  eliminate  external 
visual  cues.  The  roll  axis  system  dynamics  were  identified  and  simulated  on  a 
hybrid  computer.  The  system  characteristics  are  presented  in  Table  I.  To 
test  the  capabilities  of  the  optimal  control  pilot-vehicle  model,  it  was 
decided  to  investigate  the  effects  of  two  types  of  motion  cues  in  this 
experiment.  One  was  defined  as  the  Command  condition  because  vehicle  motion 
was  commanded  as  a result  of  following  a target  and  the  other  the  Disturbance 
condition  because  disturbance  inputs  drove  the  vehicle  directly.  Both 
conditions  were  investigated  with  and  without  the  drive  system  on,  making  a 
total  of  four  experimental  conditions.  The  block  diagram  for  the  experiment, 
presented  in  Figure  1,  shows  all  conditions.  For  the  command  condition,  the 
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TABLE  I 


MULTI-AXIS  TRACKING  SIMULATOR 
ROLL  AXIS  CHARACTERISTIC 

TRANSFER  FUNCTION;  ^ PLANT  »*  K 

u (pounds)  e(3  + 1) 

U CONTROL  ‘ 20- 

POSITION  LIMIT:  4>p  “ ‘->0  cleg 

VELOCITY  LIMIT:  <frp  « i 60  deg/ccc 

ACCELERATION  LIMIT:  $p  * ± 100  dcg/sec? 


□ a 


FIGURE  1.  System  flow  diagram. 
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disturbance  input  ( '-'DISTURBANCE^  w,3s  set  t0  zero  and  for  tbe  disturbance 
condition  the  target  input  CI’T ARGET ) was  set  to  zero.  A low  pass  filter 
with  breakpoint  at  5 radians  per  second  was  added  to  the  vehicle  making  the 
equivalent  plant  dynamics  for  all  experimental  conditions  as  given  in 
equation  1. 


Plant  Dynamics  = ^PLAN'i 


(Peg) 


"control  abs) 


In  addition,  a time  delay  of  95  milliseconds  due  to  digital  computation,  visual 
display  delay  and  an  in-line  signal  filter  existed  in  the  visual  pathway. 

In  the  command  condition,  the  task  was  to  follow  a target  aircraft  in 
the  roll  axis.  The  difference  between  the  target  roll  angle  and  the 
controlled  vehicle  position  was  provided  to  the  human  operator  on  a 9 
inch  diagonal  television  monitor.  The  inside-out  display  consisted 
of  a 1.25  inch  long  rotating  line  whose  center  was  superimposed  upon  a 
stationary  horizontal  line  as  indicated  in  Figure  2.  A 0.083  inch  perpendicular 
line  at  the  center  of  the  rotating  line  provided  upright  orientation. 

The  angle  between  the  rotating  and  stationary  line  depicted  the  difference 
between  the  controlled  plant  roll  angle  and  the  target  roll  angle.  The 
display  was  centered  in  azimuth  a distance  of  20.5  inches  from  the  controller's 
eyes.  Subjects'  sitting  heights  were  such  that  the  display  was  within  10 
degrees  of  eye  level  of  each  subject.  lor  the  disturbance  conditions,  since 
‘‘TARGET  = 0 > the  displayed  error  equalled  the  bank  angle  of  the  controlled 
vehicle  and  the  task  was  to  null  out  the  bank  angle  by  keeping  the  controlled 
vehicle  upright. 


EXPERIMENTAL  DESIGN 


With  the  vehicle  to  be  controlled  identified  and  the  four  tracking 
conditions  chosen,  the  next  step  was  to  select  task  parameters  for  the 
experiment.  The  following  constraints  and  design  goals  motivated  the 
selection  of  parameter  values: 


1.  To  achieve  face  validity,  we  desired  to  simulate  roll  axis 
dynamics  representative  of  high  performance  aircraft  in  air 
combat.  The  dynamics  of  equation  (1)  were  chosen  on  this  basis. 

2.  In  order  to  assure  that  roll  motion  would  he  well  a love  the 
subject's  threshold  of  perception,  and  to  allow  comparison  with 
a recent  study  (8),  an  RMS  bank  angle  of  10  degrees  was  desired. 

3.  Physical  limitations  on  the  roll  rate  and  roll  acceleration  of 
the  rotating  simulator  had  to  be  considered.  Specifically,  our 
goal  was  to  achieve  experimental  RMS  roll  rates  and  accelerations 
that  were  no  greater  than  1/3  the  corresponding  limits  so  that  these 
limits  would  be  reached  less  than  1%  of  the  time'. 


4.  A wide  bandwidth  of  pilot  response  was  desired  to  maximize 
our  ability  to  analyze  the  effects  of  motion  cues  on  pilot 
response  behavior;  at  the  same  time,  we  wanted  to  avoid  a 
tracking  task  that  was  unreasonably  difficult. 

5.  In  order  to  test  our  model  for  motion  cue  utilization,  we 
desired  tasks  in  which  motion  would  have  a significant  effect 
on  pilot  response  behavior. 


Experimental  parameters  that  wc  could  adjust  to  meet  these  goals  consisted 
of  (1)  RMS  amplitude  and  spectral  shape  of  the  tracking  input,  (2)  control 
gain,  and  (3)  performance  criterion. 

The  input  amplitude  was  adjusted  to  induce  vehicle  response  of  the 
desired  magnitude,  and  the  control  gain  was  adjusted  to  allow  such  response 
to  be  achieved  with  comfortable  control  forces.  A second  order  noise 
process  was  considered  for  the  tracking  input  and  the  critical  frequency  of 
the  input  spectrum  was  chosen  to  achieve  the  desired  balance  between 
measurement  bandwidth  and  tracking  difficulty. 


To  keep  RMS  response  rate  and  acceleration  well  below  the  physical 
limitations  of  the  rotating  simulator,  as  well  as  to  encourage  the  test 
subjects  (who  were  not  trained  pilots)  to  respond  in  a smooth  manner,  a 
performance  criterion  was  defined  as  the  weighted  sum  of  mean-squared  tracking 
error  and  mean-squared  vehicle  acceleration.  That  is, 


C “ o2  + W a2 

Terror  *plant 
.2 


(2) 


where  C is  the  total  "cost",  o''  the  variance  of  the  tracking  error, 

'♦’error 

and  o2..  the  variance  of  the  acceleration  of  the  vehicle  or  simulated 

♦PLANT 

vehicle,  in  the  absence  of  motion  cues. 


The  immediate  effect  of  introducing  a penalty  for  vehicle  acceleration 
was  to  limit  the  gain  of  the  subject's  response;  the  larger  the  weighting 
W,  the  lower  the  pilot  gain.  Pilot  gain , directly  influenced  overall  man/machine 
system  bandwidth,  which  in  turn  influenced  roll  rate  and  roll  accelerations 
achieved  during  tracking. 


Task  parameters  were  selected  in  the  following  way.  An  initial  set  of 
parameters  was  chosen  based  on  knowledge  gained  from  previous  experimental 
studies,  and  predictions  of  pilot-vehicle  performance  were  obtained  with  the 
pilot -vehicle  model.  Task  parameters  were,  readjusted  in  an  attempt  to  better 
meet  the  experimental  constraints,  and  the  system  was  reanalyzed.  We  iterated 
on  this  procedure  until  satisfied  with  the  expected  outcome  of  the  experiment. 
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The  optimal  control  pilot-vehicle  model  n.-od  in  this  procedure 
has  been  described  in  the.  literature  (9-12)  and  is  reviewed  briefly  in 
a companion  paper  (13).  Independent  pilot-related  model  parameters  were 
held  fixed  throughout  this  procedure  at  values  obtained  from  previous 
analysis.  Specifically,  time  delay  was  set  at  0.17  second,  the  "motor 
time  constant"  (a  first-order  lag  associated  with  pilot  response)  was 
0.1  seconds,  and  the  "noise/signal  ratio"  (to  account  for  pilot  response 
randomness)  was  set  at  -20  dB.  Visual-only  tracking  was  represented  by 
considering  only  tracking  error  and  tracking  error  rate  in  the  set  of 
informational  quantities  available  to  the  pilot.  Roll  angle,  roll  rate, 
and  roll  acceleration  of  the  simulated  vehicle  were  added  to  this 
informational  set  to  account  for  the  presence  of  motion  cues. 

As  a result  of  this  iterative  design  process,  the  following  task 
parameters  were  selected.  The  force  stick  gain  was  adjusted  to  produce 
10  degrees/second  vehicle  roll  rate  for  one  pound  of  force  measured  at 
thumb  height  on  the  control  grip  and  the  cost  weighting  W (equation  2) 
was  set  to  0.1.  In  addition,  both  the  target  and  disturbance  inputs 
were  constructed  from  13  sinusoids  whose  amplitudes  were  selected  to 
simulate  random  noise  processes  having  power  spectral  densities  of  the  form 

«>  " I Ji— ...  J2 

(ju  + ui)- 

where  toq  was  1.0  rad/sec  for  the  target  input  and  2.0  rad/sec  for 
the  disturbance  input.  Input  amplitude  was  adjusted  to  provide  an  RMS 
target  input  of  10  degrees  and  an  RMS  disturbance  input  of  14  deg/sec. 

In  order  to  prevent  subjects  from  learning  the  input  waveforms  during 
the  experiment,  a random  number  generator  was  used  to  vary  the.  phase 
relationships  of  the  input  sinusoids  from  one  experimental  trial  to 
the  next. 

EXPERIMENTAL  PROCEDURE 

Six  healthy  college  students  between  18  and  25  years  of  age  were 
used  for  the  experiment.  Subjects  tracked  each  condition  each  day. 

Tracking  under  each  condition  was  considered  one  run.  Each  run  lasted  . 

165  seconds  and  the  four  condi tions  or  runs  were  presented  in  a random 
order  each  day.  At  the  end  of  each  run,  subjects  were  presented  their 
tljree  performance  scores  for  that  run:  t;otal  cost  C,  error  variance 
o.  and  weighted  acceleration  0.1  a-  . They  were  instructed 

error  'plant 

to  minimize  the  total  cost  C.  In  addition,  they  were  told  that  it  was  the 
sum  of  the  other  two,  that  the  error  score  was  related  to  how  much  error 
they  allowed  and  that  the  acceleration  score  was  related  to  how  smoothly  they 
tracked.  They  were  not  told  predicted  scores,  nor  were  they  told  how  to 
divide  their  total  score  between  error  and  acceleration.  To  maintain 
subject  motivation,  subjects  were  also  made  aware  of  each  other's 
performance  scores.  Each  subject  wore  a flight  helmet  with  intercom 
capability  while  performing  the  tracking  task.  The  subject  was  permitted 
to  perform  the  task  briefly  prior  to  each  scored  run  in  order  to  adjust 
mentally  and  physically  to  the  tracking  task. 
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Performance  scores  were  plotted  dally  in  order  to  evaluate  subject 
and  group  performance.  Once  the  error  scores  Indicated  that  the 
subjects  had  "learned"  the  tracking  tasks  for  all  experimental  conditions, 
tracking  was  continued  for  another  eight  days  and  time  history  data  was 
collected  for  subsequent  analysis.  From  these  last  eight  days  of  runs, 
performance  scores  were  computed  for  each  subject  for  each  condition, 
making  a total  of  48  measurements  per  condition.  For  purposes  of  comparing 
the  experimental  results  to  predicted  values  for  each  condition,  the 
results  of  the  six  subjects  were  averaged  together. 

RESULTS  AND  DISCUSSION 

Once  subject  training  had  been  accomplished,  data  was  collected 
for  eight  days  for  all  subjects.  Training  was  considered  completed  when 
subject  performance,  as  measured  by  total  cost  C for  all  conditions,  had 
reached  asymptotic  levels. 

From  the  collected  data,  various  system  parameter  values  were 
computed  and  averaged  together  across  days  and  subjects.  The  experimental 
; results  along  with  the  predicted  model  values  are  given  in  Table  II  for 

| the  command  condition  and  Table  III  for  the  disturbance  condition.  The 

experimental  values  include  the  mean  and  standard  deviation  resulting  from 
averaging  together  the  six  subjects'  results.  From  Tables  II  and  III 
we  can  see  that  the  model  predictions  arc  quite  accurate,  28  of  the  36 
predictions  are  within  one  standard  deviation  of  the  means  of  the  experimental 
values  and  the  remainder  are  within  two  standard  deviations  of  the  mean. 

To  better  compare  predicted  and  experimental  results  across  conditions, 
these  values  are.  also  presented  graphically  for  total  cost  (PERFORMANCE  SCORE) 
and  pilot  input  (RMS  CONTROL  FORCE)  in  Figure  3,  for  plant  position 
(RMS  4>  FI  ANT)  and  system  error  (RMS  S’  ERROR)  in  Figure  4,  and  plant  velocity 
( RMS  4>  PLANT)  and  acceleration  (RMS  "i  PLANT)  in  Figure  5.  Experimental 
conditions  are  indicated  on  the  abscissa  of  each  giaph ; C/M  indicates  the 
command  with  motion  condition,  C/S  command-static,  D/M  disturbance-motion  and 
D/S  Lhe  disturbance-static  condition.  From  Figure  3 we  see  that  with  the 
model  we  were  able  to  predict  total  performance  score  and  the  control  force 
the  pilot  used  for  the  four  conditions.  The  same  trends  can  be  observed  in 
Figures  4 and  3 for  the  vehicle  motions  and  the  error  the  pilot  allowed. 

The  experimental  results  also  indicate  that  our  design  goals  were 
achieved.  One  of  the  requirements  was  that  the  control  tasks  not  be 
unreasonably  difficult.  The  control  forces  used  by  the  subjects  (Figure  3) 
indicate  that  the  tasks  were  not  excessively  difficult  to  control  and 
that  the  forces  were  within  the  design  region  of  0.3  to  1.5  pounds.  To 
insure  that  the  roll  motion  would  be  well  above  pilot  thrcshol ds , we  desired 
an  RMS  bank  angle  of  approximately  10  degrees.  From  Figure  4 we  see  that 
this  requirement  was  also  met.  For  the  disturbance  motion  case,  the 
subjects  were  able  to  reduce  the  bank  angle  error  below  wiiat  the  model 
predicted.  Also  for  the  static-disturbance  case,  the  model  predictions  were 
less  than  the  experimental  results.  As  mentioned  earlier,  there  existed 
a 95  m3  time  delay  in  the  visual  loop  that  wan  not  present  in  the  motion  cue 
loop.  Model  predictions  did  not  include  the  presence  of  the  visual  time 
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TABU',  II 

PREDICTED  VERSUS  EXPERIMENTAL  RESULTS  FOR  THE 
COMMAND  CONDITION 


VARIABLE  UNITS  , STATIC  MOTION 


PREDICTED 

EX PER  IMF 

NTAL 

PREDICTED 

KXPERIMENTAI 

MEAN 

la 

MEAN 

la 

Total 

Coat 

— 

67.0 

72.8 

8.9 

53.1 

66.1 

7.5 

Coat 

cn  *E 

— 

52. A 

50.7 

11.9 

45.5 

48.2 

11.1 

Coat 

on  $p 

— 

14 . 6 

22.2 

9.7 

7.57 

17.90 

8.64 

a 

u 

pounds 

0.730 

0.754 

0.120 

0.516 

0.669 

0.156 

deg 

10.1 

9.2 

0.9 

6.37 

7.63 

1 .25 

"♦p 

deg/sec 

6.86 

7.10 

1 .11 

4.84 

6.74 

1.69 

X 

deg/sec^ 

12.1 

14.6 

3.1 

8.70 

12.9 

3.51 

X 

deg 

7.24 

7.06 

0.74 

6.75 

6.90 

0.78 

X 

deg/sec 

11.7 

11.9 

0.7 

10.7 

11.8 

0.79 
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TABU’.  Ill 


PREDICTED  VERSUS  EXPERIMENTAL  RESULTS  FOR  THE 
DISTURBANCE  INPUT  CONDITION 

VARIABLE  UNITS  STATIC  MOTION 


PREDICTED 

EXPERIMENTAL 

- 

PREDICTED 

EXP  ERI.’- 

T.NTAL 

MEAN 

lo 

MEAN 

]c 

Total  Cost  

178 

197 

29 

91.6 

78.6 

15.5 

Coat  on<{>£  

86.2 

81.6 

33.4 

37.2 

20.5 

15.4 

Cost  on^p  

91.5 

115 

36.7 

54.4 

58.1 

9.2 

0 

u 

pounds 

1.56 

1.55 

0.14 

1.49 

1.49 

.08 

\ 

deg 

9.29 

8.83 

1.80 

6.1 

4.2 

1.5 

% 

deg/aec 

12.1 

11.9 

1.3 

7.76 

6.81 

1.41 

°A 

deg/sec2 

30.3 

. 33.6 

5.0 

23.3 

24.0 

1.9 

— @- 
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performance  and  control 


(oaa)  aouu3  <s>  swa 


FIGURE  4.  Model  and  experimental  results  compared  for  vehicle  position 
and  system  error. 


FIGURE  5.  Comparison  between  model  and  experimental  results  for 
vehicle  velocity  and  acceleration. 
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delay.  These  circumstances  may  account  for  the  differences  between  the 
model  predictions  and  experimental  results.  The  physical  limitations  of  the 
simulator,  namely  a velocity  limit  of  60  deg/sec  and  an  acceleration  limit  of 
100  deg/sec^,  had  to  be  considered  during  motion  tracking.  Our  design  goals 
were  to  achieve  experimental  RMS  roll  rates  and  accelerations  that  were  no 
greater  than  1/3  the  corresponding  limits.  The  experimental  results 
(Figure  5)  indicate  that  these  goals  were  satisfied. 


As  stated  earlier,  the  motion  sensitive  aspects  of  the  model  were 
developed  for  experimental  conditions  different  from  those  investigated 
in  this  study  and  a different  simulator  with  narrower  bandwidth 
vehicle  dynamics  (8).  These  facts  further  emphasize  the  usefulness 
of  the  predictive  capabilities  of  the  model.  The  next  step  in  the  study 
was  to  determine  what  model  parameter  adjustments  were  needed  to  improve 
the  match  to  the  data.  This  is  the  subject  of  a companion  paper  in  these 
Proceedings.  By  readjusting  the  model  parameters,  we  hope  to  gain  additional 
insight  into  how  the  pilot  utilizes  motion  information. 


CONCLUSIONS 


The  major  objectives  of  our  experimental  program  have  been  (1)  to 
investigate  the  usefulness  of  the  model  as  an  experimental  design  tool, 

(2)  to  demonstrate  the  ability  of  the  model  to  predict  the  influence  of 
motion  cues  on  pilot-vehicle  performance  for  different  tracking  tasks  and 

(3)  to  provide  a data  base  from  which  we  could  improve  our  understanding 
of  how  the  pilot  utilizes  and  is  effected  by  motion  cues.  In  conclusion, 
we  feel  the  results  of  this  experiment  demonstrate  the  usefulness  of  the 
predictive  optimal  control  pilot-vehicle  model.  With  the  model,  we  were 
able  to  predict  pilot-vehicle  response  for  the  various  motion  cue  conditions. 
In  addition,  by  making  use  of  the  model,  the  experimental  design  process 

was  not  only  simplified,  we  were  assured  that  useful  data  could  be  collected. 
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